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AN EXTENSION OF THE RELIABILITY CONCEPT TO VECTOR VARIABLES 



Walter Kristof 
Abstract 

The coefficient of precision type of reliability originally defined, 
for scalar variables is generalized to vector variables and named vector 
reliability. The new coefficient ranges from zero to one. Additional basic 
properties of vector reliability are derived. Vector information is defined 
as a simple function of vector reliability. A number of properties of 
vector information are demonstrated in order to justify its use as a measure 
of information contained in a vector variable. Two applications are appended 
by way of illustration. 



AN EXTENSION OF THE RELIABILITY CONCEPT TO. VECTOR VARIABLES - * - 

Definition and Discussion of Vector- Reliability 
A familiar definition of the reliability of a scalar random variable 
X with true score T and error f/ , X = T + E , is given by the coefficient 
of precision, 

(!) P* = 4 /<% . 

If is to express an intrinsic property of variable X and if this vari- 
able represents measurement along an interval scale, then should be 

independent of linear transformations of X . This is indeed the case if we 
introduce the rule that transformation Y = aX+b, a / 0 and b constants, 
shall imply the true score transformation U = aT + const. This rule is cer- 
tainly in agreement with our intuitive notion of true score. 

In this note we will first extend the reliability concept to vector vari- 
ables along the lines indicated recently by Conger and Lipshitz ( 1971 ). In 
this new presentation, the use of the covariance matrix instead of the correla- 
tion matrix leads to desirable invariance properties. Reliability is explicitly 
defined in terms of population characteristics. In addition, an analog to the 
notion of information in vector variables will be developed and presented. 

Let us again employ the classical test theory model. Hence, for vector 
variables , X = T + E . It will be assumed that T and E are statistically 
independent. As a generalization of (l) we define the reliability of 
vector X as 
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column vectors X and T having means and , respectively. 7^ is 

the variance -covariance matrix of X and is assumed to be positive definite. 

The denominator in (2) is simply the number of components of X , m say. 
The numerator equals tr 2^1^ , being the variance-covariance matrix of 

T . Hence (2) can be rewritten as 



(3) 



**X m tr ^T^C 



-1 



Denoting the reliability of the i -th component of X by and writing 



h = > 



, we have also 



m 
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(4) \ ■ 1 - 5 ^ (1 - p i> 

when the components of E are uncorrelated. 

The following is seen to be true: 

(i) If we adopt the transformat ion rule that a linear transformation 
Y = AX + B , A nonsingular, implies the transformation of true scores 
U = AT + C , C any constant vector, then remains unchanged under 
transformation of X as follows from (3). Hence p^ = p^ . 

(ii) Independence of true score and error is preserved under such linear 
transformations of X. However, initially uncorrelated errors may become 
correlate^. 

(iii) We have 0 < p^ < 1 . For, nonnegativity of p^ follows at once 
from (2) since is positive definite. Further, there is always a linear 

transformation of X that will lead to uncorrelated errors. At the same 
time, p£ remains unaltered according to (i). Hence, p^ < 1 as follows 
from (4). 
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(iv) Assume that the components of E axe uncorrelated and < 1 
for all i . Then vector reliability equals the average component 

reliability, p^ = Ep^/m , if and only if the components of X are uncor- 
related. If the components of X are not all uncorrelated, however, then 
Pjj < EP i /m . Proof: 

Let us deal with the first part of the assertion first. We derive from 
(U) that p^ = Ep^/m is equivalent to 



(5) E(i - cr ii a 11 )(l - p.) = 0 . 

i 

Sufficiency of the condition follows upon noting that (5) is satisfied when 

o- 11 = l/o\ ^ . As to necessity, let have a canonical decomposition 

= PAP * where P = llp^ll is orthogonal and A is positive diagonal with 

X. in the j -th diagonal position. Then 
J 



m 2 

a. . = E p. .X . 
ii . =1 P 1D 3 



m 
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The Cauchy-Schwarz inequality yields 



(6) 



I ii^ v 2 i 

J ^ii* - = p ij = 1 



D=1 



consequently 1 - cr^cr 11 <0 for all i . But 1 - Pj^ > 0 for all 



In 



order to satisfy (5) it is necessary that always 1 - cr^cr = 0 . Therefore 

* * 2 
cr 11 = l/<r for all i which implies that is diagonal. 
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An explicit proof of this implication could be written down at once and 
has been omitted. 
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As to the second part of the assertion, it now follows from the above 
Cauchy -Schwarz inequality (6) that there must be some k for which 

Thus 



, kk ^ n 

1 - "Hf < 0 



E(1 - a ii cr ii )(l - 0 i ) < 0 



which is equivalent to < Ep^/m . This completes the proof . 

Hence we see that p^ will be at most equal to Ep^/m . 

(v) p^ = 0 is equivalent to = 0 for all i . This fact is easily 
established by using (5) and considering that E,p is positive semidefinite 



and is positive definite . 



(vi) Pjj. = 1 is equivalent to = 1 for all i provided that 
remains positive definite. For, = 1 for all i implies E^, « , 

hence fy = 1 by (3). On the other hand, there is a linear transformation 
of X to Y that w ill lead to uncorrelated errors in Y and leave p^ = Py 
according to (i). Denoting parameters which refer to Y by superscript (Y) 
we have from (4) 



r .(v) ff ii(2) (1 . P (D) . o 



r°ii 

1 



Because > 0 we conclude that p(-^ = 1 for all i . Hence all 



components of Y are error-free. The same must then apply to components of 
X because these are linear combinations of the former. Therefore p^ - 1 
for all i . 

(vii) Statement (vi) depends crucially on ^ remaining positive definite. 
This becomes evident when we consider the following case. Let the components 
of X represent parallel forms of the same test with common reliability p < 1 
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Assume that errors are uncorrelated. 2^ is positive semidefinite. We find 
that 

= [l + (m - 2)p]/[l + (m - 2)p - (m - l)p 2 ] 
and obtain from (k) the result 

(7) Pjj = p/[l + (m - l)p] 

The derivation of this formula breaks down when p = 1 , i. e. , when is 

not positive definite. However, we can still obtain 



lim Py = l/i 
P*1 - 



1 
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On the other hand, if the components of X do not represent par all el (but 
possibly nearly parallel) forms and 2^. stays positive definite then p^ -*• 1 
implies p^ = 1 according to (vi). 

(viii) There is a certain relationship between p^ and the maximal reliabil- 
ity of a linear composite of the components of X . This maximal reliability 
is given by the largest eigenvalue of • For a derivation and additional 

references see Lord and Novick, 1968, p. 123. In contrast, p^ is the average 
of all eigenvalues of as follows from (3)« 

This concludes our enumeration of basic properties of p^ . 

2. Definition and Discussion of Vector Information 

2 2 

The reliability p = o^/a^ of a single variable X can be viewed as a 
measure of the amount of "true information" contained in a typical observed 
value of X • The scaling is such that 0 < p < 1 . It appears that this 
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concept can be natur ally extended to vector variables X with m > 1 
components. The quantity mp^ exhibits features that we would require 
such a gener ali zed measure to possess. Let us restrict ourselves to the 
nondegenerate case when is positive definite. We find the following. 

(i*) wt p remains unchanged under nonsingular linear transformations 
of X when the transformation rule given in (i) is adopted. 

(ii 1 ) 0 < mp£ <m according to (iii). 

(iii') mp^ = 0 precisely when T is a constant as follows from (v). 
Hence X may be regarded as containing only error. 

(iv* ) mp£ = m precisely when E is a constant as follows from (vi). 
Hence X may be regarded as containing no error. 

(v* ) Let the (m^ + mg) -component vector X be partitioned into 
subvectors X-^ end Xg with m^ and mg components , respectively, 
m = + mg . If, for general m , the quantity mp^ is to be interpreted 

in the sense of a measure of "information" contained in the m -component 
vector X , then we should require the property of subadditivity, 



W + “a>% 2 + >V>c 2 - 

when errors are not correlated across X^ and Xg . The equality sign 
should be expected to hold when, in addition, X_^_ Xg are independent 

of each other, i.e., when these subvectors supply independent information. 

We w ill prove that these requirements sire indeed satisfied. Without loss of 
generality we suppose that m^ > mg . 

Let us perform two separate linear transformations, X^ Y^ and Xg -> Yg , 
such that the psirtitioned variance-covsiriance matrix of Y , where Y results 
from the combined transformation X Y , tsikes on the form 
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Here 1^ and Ig are identity matrices of orders and mg , respectively. 

Matrix r is of order m^ x mg and contains in its truncated diagonal the 
canonical correlations 7 ^ between X^ and Xg , i = 1 , ...,mg indicating 
the position, and zeroes elsewhere. 

The above transformation X -*■ Y carries at the same time the original 
true score vector T in X = T + E. into a new true score vector U composed 
of subvectors and Ug corresponding to Y^ and Yg . The variance - 

covariance matrix of U may then be partitioned accordingly, 

V 

with A = Ha^H , B = ||b i ^ || , C = Hc^H . Let us partition Z ^ 1 similarly, 

F ! G 
1 

G’ *, H 

I 

Taking the special form of Ey into account we find that F = llf^ll an< ^* 

“ 2-1 

H - M are diagonal matrices with f^ = (l - 7 ^) for i < mg and 

f = 1 for m 2 < i < , h^ = (l - /f )" 1 with i < mg . The x m 2 

2 -1 

matrix G = lls^ll contains in its truncated diagonal ®ii ” “^i^ ” ^i^ 
and zeroes elsewhere. 
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Invoking (i* ) we obtain 
(8) (n^ + = (i\ + m 2 )p Y = tr 

= tr AF + 2 tr CG* + tr BH 



The required traces are easy to evaluate due to the initial transformation 
X -*• Y . We get 



“l 



“l 



(9) tr AF = E a u f ii = E^ + E 



m 2 
“2 ?i 



i=l xx J " L i=l xx i=l 1 - 7. 



2 ii 

9 



m 2 y\ 

m,Py + E 5 Pii * 

~1 i=l 1-74 



“l 

Here we have used the fact that E a.* = tr AI^ equals n^Py and hence 

i=l “1 

als o m^ due to (i')« By we designate the reUability of 

the i -th component of . Y. . In complete analogy, 

“2 y\ 

(10) tr BH = HUPy. + E 5 P04 

-2 i=l 1 - y. 



where p» denotes the reUability of the i -th component of Yg . Finally, 

“2 74 “2 y \ 

(U) 2tr CG* = -2 E * c.. = -2 E £ • 

~ i=l 1 - 7i i=l 1 - y± 

In taking this step we assume that errors across X^ and Xg and so across 
Y^ and Yg are uncorrelated, thus c^ = 7^ • Insertion in (8) yields now 



2 

®2 V 

(12) = V*. ♦ »a(V - S : 5 < 2 * Pii - p 21> 

— —1 -« 1=1 I-7. 



Therefore we have indeed subadditivity, 



(15) K + ">a)t)c - "aV * “2%, • 

We will generally expect that 2 - p * ^ - Pg^ > 0 . Hence the equality sign 
in ( 13 ) will hold precisely when all canonical correlations 7 ^ are zero, 
i.e., when all correlations between components of X^ and components of 
Xg are zero. This completes the proof of (v' ). 

(vi* ) Let the (m^ + mg) -component vector X be partitioned as in (v* ). 
If, for general m = + mg , the quantity mp^ is to be interpreted in the 

sense of a measure of "information" contained in X , then we should require 
the property 

(n^ + mgjp^ > max (m^ , mgp^ ) . 

— —1 —2 



We will prove this property when errors are not correlated across X^ and 

Upon making the same transformation of X^ and Xg as in (v* ) and using 



Xg . 



(Ik) 



mg 



mpy = tr BIp = £ Pp. 
^2 i=l 



we obtain from (12) the difference 6 , 



(15) 6 = (n^ + mgjp^ - 



mg 

S p 2i 
i=l 



\ A ( 2 " p ii " p 2i) 

i=l 1-7. 
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Since y^ < p .* . we have farther 



(l6) 



“2 

5> Z p* 
i=l 1 



g 2 P li P 2j 



1=1 1 ‘ p ii p 2i 



( 2 “ P{i - P^) 



For 0 < pj^ < 1 the expression 



p li p 2i (ox x \ 
1 ' p ii p 2i 2 " Pli ’ p2i 



is a nondecreasing nonnegative function of p *,. and tends to p^ when 
p| . 1 . Therefore 



( 17 ) 



*2 “2 Pk 

5 > Z P^ - Z i . b (1 - P^ ) = 0 , 



i=l 






2i ; 



i.e., (m^ + nigJPjj > m^p^ . An analogous derivation holds when the roles 

of m^ and nig are interchanged. Consequently, 

( 18 ) (m^ + MgJPjf > max (m^ , m^ ) 

— —1 —2 



as required. This completes the proof of (vi 1 ). 

Properties (i* ) - (vi*) will sufficiently justify our use of 



,-l 



mp~ = tr as a measure of information contained in X 



3. Illustrative Examples 

X 

Example 1 ; The following application uses real data and will serve to 
illustrate the meaning of p^ and mp^ . It is based on Wechsler's (1958, 
pp. 100-103) publication of data on the WAIS for the age group 25-3^ years. 



? This example is different from the one used by Conger and Lipshitz (1971), 
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The WAIS is composed of six verbal tests apd*five performance tests. The 
examiner will typically attempt to interpret differences in a subject's per- 
formance on the two subbatteries. We may therefore ask, for example, to 
what extent the two subbatteries do furnish independent information. 

The WAIS contains the following subtests: 1. Information, 2. Com- 

prehension, 5. Arithmetic, 4. Similarities, 5. Digit Span, 6. Vocab- 
ulary, 7* Digit Symbol, 8. Picture Completion, 9« Block Design, 

10. Picture Arrangement, 11. Object Assembly. However, the performance 
test Digit Span will not be included in the analysis because the reliability 
of this test is not reported in Wechsler's (1958# p. 103) table for the age 
group considered. Hence we will be dealing with a verbal battery of six 
tests (tests 1-6) and a performance battery of four tests (tests 8-11) only. 

The following table contains the intercorrelations of the tests, the 
diagonal gives the reliabilities. The first two digits after the decimal 
point are listed. 




O 



The entire table can be taken as representing 1^, with m = m^ + mg = 10 , 
m^ = 6 , mg = 4. Matrix 2^ results from substituting ones for the relia- 
bilities in the diagonal. Matrices L , L , L and are obtained 

— 1 ~±2 -1 -2 

as portions. 

We will assume independence of errors of measurement across tests. Upon 

inversion of 2^ , 2^ and 2^ we use (4) and get the following vector 

reliabilities: p^ = .5445 , p^ = .6139 and p^ = .5226 . The correspond- 
— —1 —2 

ing amounts of information become mp^ = 5*4445 , m^p^ = 3*6855 and mgP^ = 

— — 1 ~ 2 

2.0904 . 



m^p^ is I.7610. This is the portion of the 
information represented by the total WAIS battery that is not accounted for 
by the verbal subbattery and hence is added by the performance subbattery. 

This added amount is contributed by four tests. So, on the average, each of 
them contributes less than one half of the theoretically possible amount. 

The information contained in the total battery comes from ten tests. 

Hence, on the average, each of them contributes a little more than one half 
of the theoretically possible amount. Figures of this sort may help in assess- 
ing a battery of tests. 

Example 2 ; Suppose we wish to measure a trait by using a test whose 
reliability is p < 1 . In order to increase accuracy we administer m 
parallel forms of the test with uncorrelated errors. We expect that, on the 
average, the sum of the m measurements will be more accurate than any single 
measurement. In fact, the Spearman-Brown formula of classical test theory 
tells us that the reliability of the sum will be mp/[l -s- (m - l)p] which 
exceeds p . This is, at the same time, the amount of information contained 
in the sum, the sum being regarded as a new variable. 



The difference 



m Px 



Using the theory developed in this paper we see that taking the sum is 
indeed optimal. We axe extracting the maximum, namely, the total amount of 
information contained in m parallel measurements. For, according to (7), 
this total is mp^ = mp/fl + (m - l)p] . 
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